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ABSTRACT 

There is a need for improved educational indicators 
in order to determine the nature of the major problems facing 
American education. Based on the experience of the National 
Commission for Excellence in Education, this paper tries to provide 
the basis for identifying a preliminary set of indicators that would 
be useful to policymakers and that are not currently available. In 
the input stage of the educational system the indicators would be: 
amount of time allocated to education, content, standards and 
expectations, teaching as a practice and as a profession, and 
leadership and support. Factors identifiei in the output stage are 
achievement and retention. There are certain kinds of information 
which are essential for defining variables but not available, such 
as: number of students who know basic academic concepts, achievement 
of students compared to students in other countries, ability of 
students to solv. complex problems, trends in measuring educational 
achievements, and trends in achievement facing different challenges. 
Reasons are offered to explain why some of the indicators are not 
available. (JAZ) 
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When the National Commission on Excellence in Education prepared 
its report, A Nation At Risk , it was concerned very much with 
educational indicators. The Commission was charged, in part, with 
determining the nature of the major problems facing American 
education. To do this, the Conmission needed to examine the evidence 
that was avail£U>le of the health of the country's educational system. 
The products of this effort are two sections in particular in A Nation 
At Risk ; a section called "Indicators of the Risk," which is a 
c(»qpendium of the major pieces of evidence the Commission found 
concerning the quality of American education; and sections of 
"Findings" in the areas under which the Commission organized its 
analysis and recommendations — time, content, expectations, teaching, 
and leadership. 

The figures cited in these sections encompass student achievement 
in basic skills and academic subjects, rates of functional illiteracy, 
trends in the amount of homework assigned to students, figures on 
average teacher salaries, coiq>arison8 of the time spent in school by 
U.S. students with the time spent by their counterparts in other 
countries, and so on, but the Commission's experience revealed perhaps 
as much about the inadequacy of our educational indicators as it did 
about the inadequacy of our educational programs . The "Findings'* and 
"Indicators of the Risk" cited by the Conmission were the best 
evidence available on in^rtant aspects of our educational system, but 
there was much more information that should have been included in 
these sections that singly was not available. The Ccmunission could 
say nothing, for example , about the general skill of American teachers 
in presenting subject matter and conducting lessons. No figures were 
available to indicate whether functional illiteracy was increasing or 
decreasing over time. Comparisons of the achievement of U.S. students 
with that of students in other countries was fifteen years old for 
some subject areas. Finally, detailed information £a>out what high 
school students actually studied and knew In subjects like science, 
social studies, mathematics, literatiire, and the humanities had never 
been collected or reported on a nationally-representative sample. 

At least one indicator — the courses typically taken by high 
school students during their four years in high school — was felt to be 
so important that the Commission arranged for its collection, having 
found it was not available from any other source. The resulting 
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study revealed that students took alarmingly few courses in the 
academic subjects. This finding led the Commission to recommend that 
a minimum number of courses be required in the academic "basics** — 
English, science, social studies, and mathematics, as well as computer 
science and (for the college bound) foreign languages — in order for 
students to be granted high school diplomas, and over the past 3 or 4 
years, most states have increased these course requirements. This 
experience underscores the importance of this information, a level of 
importance which is not consistent with the inattention that this 
paurticular piece of information has been given among our educational 
data- col lection activities. 

Significant efforts are underway to improve our educational 
indicators. The Department of Education initiated an ** indicators'* 
project that has, thus far, provided a useful taxonomy for educational 
indicators and a description or compendium of twenty of the key 
indicators in this taxonomy that are currently available. A periodic 
report of these key indicators was initiated by NCES in 1985. 
(Indicators , 1985) The Department also initiated a one-sheet chart 
displaying key educational information for each of the states. (U.S. 
Department of Education, 1984, 1985) The Chief State School Officers 
as a group have endorsed (and taken the initiative for con^iling) 
educational assessment data, including school outcome data, on a 
state-by-state basis ^ and the National Assessment of Educational 
Progress has taken steps to enable states to piggy-back on the 
National Assessment to collect state-by-state achievement data for 
purposes of self-monitoring and comparison. 

Despite these efforts, a crucial step remains to be taken. This 
step is to consider systematically the information that we need but do 
not have about education, identifying and planning those indicators 
that should be added to, or improved among, our current set. 

Vftiat I will attempt to do in the remainder of this paper is to 
establish a framework for organizing and considering educational 
indicators, review which of the indicators in this framework are 
currently available, and identify some of the important indicators 
that, based on the experience of the Commission on Excellence, are not 
available . 

A Framework for Educational Indicators 

The concept of an educational indicator involves two elements. 
Each of these has been addressed by other analysts, but they have not 
generally been considered together. First, cm indicator describes a 
variable in the educational system or educational process. The 
National Commission on Excellence in Education organized these 
variables under an input-output model, with an emphasis on the inputs 
of education: 
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Figure 1 

Framework for Variables in Education Used By 
The National Commission on Excellence in Education* 
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*The variables shown are intended to illustrate the categories in this 
framework; they are not necessarily all of the variables considered by 
NCEE. 
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The Education Department's Indicators Project uses a similar 
scheme, describing education as a process, but it classifies the 
indicators under ''outccxnes," "resources," and the "context of 
education," or those factors such as Instructional climate and parent 
support that have an effect on the success of the educational process* 

These schemes are alternatives that attempt to serve the Scune 
purpose: to model the educational process or system. One must decide 
if the models ara interchangeable or whether they differ, and, if they 
differ, whether one of them is more valid, comprehensive, or 
parsimonious than the others. 

The second element involved in the idea of an educational 
indicator is the purpose to which the indicator is applied (Selden, 
1984; and Smith, 1984) • As we have pointed out, indicators can be 
compared to absolute standards of how we want the educational system 
to perform. They can be \ised to coiqpstre the performance of our system 
with the performance of other systems. Or, they can be used to see 
how our system is doing in relation to how it has performed in the 
past. The Coimnission looked at evidence of the quality of American 
education by conparing it in scHoe cases to absolute standards (23 
million illiterate adults is more than we should accept in this 
society) ; by coii%>arlng it with similar evidence pertaining to 
education in other countries (twelfth graders in the U.S. do fair to 
poorly in mathematics when ranked among twelfth graders from developed 
countries) ; or by cosapaxing it with evidence of how we have done in 
the past (17-year-old8 are less able to draw inferences from their 
reading now than they were in 1970) . These are the three major types 
of analysis that can be made with an educational indicator. 

Smith explains these functions quite ably and how they turn 
simple variables about education into indicators which are useful for 
setting policy. He also points out that indicators can be analyzed in 
conjunction with one another to explore how the educational system 
works (and how policy decisions might affect it). For exan^le, while 
the aptitude of students entering teacher-preparation programs has 
declined, so have average teacher salaries, in real dollars, 
suggesting that there may be a relationship between these trends. 

Given these two notions essential to the concept of an 
educational indicator (the features of the educational system that 
they describe, and the purposes for v/hich they are examined) , one is 
tempted to array indicators in a matrix: 
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Figure 2 



Matrix of Educational Variables and 
The Purposes to Which They Can be Put As Indicators 



EDUCATIONAL SYSTEM 
Inputs Outputs 

PURPOSES 



Comparison with 
a stcuidard XXX XXX 

Comparison with 
another system XXX XXX 

Comparison with 
past performance XXX XXX 

Analysis of how 
the system works XXX XXX 
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In t:hi8 matrix, each characteristic of the educational system 
could be reviewed for any or all of the four purposes shovm# and any 
type of comparison or analysis could be made on any of the 
characteristics of the system. In real practice, it would not be 
useful or appropriate to make all of these analyses. Some cf the 
inputs cuid outputs of the educational system have no inherent, 
absolute standards associated with them, or they may have no scandard 
to which we would all agree. In these cases, reference to other 
systems or to past performance may be the only reasonable basis for 
interpreting the indicator. Exanyples of this situation might be 
indicators of the ability of teachers to present lessons before a 
classrocHn, or the fiscal effort commiinities put forth to pay for their 
schools. In these cases, we really have no basis for deciding the 
level on the indicator that is desireable or acceptable. In other 
cases, we may have set absolute goals for the U.S. educational system 
that are one-time efforts, or that are unique to the U.S. system and 
do not pertain to other systems, such as extending education to the 
disadvcuitaged over a certain period of time. In these cases, we %K>uld 
concentrate on our attainment of the goal and would not (perhaps could 
not) review progress in relation to past performance or to other 
countries. 

With these caveats in mind, one can imagine a list of variables, 
and purposes to which they can be interpreted, that would constitute a 
comprehensive and idealized"* list of educational indicators. The 
question before us is: Among the most important indicators in such a 
list, which ones are available to us and which ones are not? 



Educational Indicators — ;^at Do We Know and Not Know? 



The experience of the National Commission on Excellence in 
Education gives us a basis for identifying a preliminary set of some 
of the indicators that would be useful to policy makers, and that are 
not now available. To do thiS/ let us go through the taxonomy of 
indicators that the Commission considered, to identify at least some 
of the cuialyses that I feel the Commission was not able to make, 
because the necessary data did not seem to be availcible or useful. 

INPUTS 

1. Time — The amount of time alloc ted to education, and the use of the 
time allocated by teachers and students. 

What the Commission reported concerning time ; 

^ That, in the U.S., the typical school year is 180 days and the 
typical high school day is 6 hours, compared with England and other 
industrialized countries, where it is not unusual for high school 
students to spend 8 hours per day, 220 days per year in school. (A 
Nation At Risk , page 21) 

^ That, out of 30 hours of student attendance in school each week, the 
average school provides 22 hours of academic instruction, and some 
schools provide only 17 hours. (Risk, page 22) 
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^ That, because of differences among teachers in their ability to 
manage classroom time, some students may receive only one-fifth the 
instruction in reading comprehension tliat is provided to other 
students. (Risk , page 22) 

^ That, in most schools, students are not taught how to manage their 
time, through planned and systematic instruction in study skills. 
(Risk , page 22) 

® And that students in the U.S. are asked to spend too little time on 
homework, and the time they spend on it has declined. ( Risk , page 19) 

What the Commission could not report . . . 

... concerning the volume of time allocated to schooling ; 

® How much states and local school districts vary in the length of the 
school day or school year: for exan^le, how many districts provide 
five, six, or seven hours of attendemce per day, or what proportion of 
districts offer more or fewer than 180 days of instruction per year? 

® Whether the length of the typical school day or school year has been 
increasing or decreasing over time in this country. 

. . . concerning the use of time by teachers and students ; 

® Whether (and how much) schools around the country, at both the 
elementary and secondary levels, vary in the proportion of allocated 
time that they actually schedule for academic instruction. 

® How U.S. schools compare with schools in other countries in the 
proportion of allocated time that they schedule for academic 
instruction. 

® How U.S. teachers compare with teachers in other countries in their 
ability to manage classroom time. 

® Whether teachers have become more or less proficient over the years 
in their ability to engage students in academic learning. 

® Whether students in other countries are taught work habits and study 
skills better than students are taught these skills in this country. 

® How much variation exists in the teaching of study skills among 
schools "^.i this councry, and whether the variation is systematically 
related to characteristics of the schools. (Do certain types of 
schools provide this kind of instruction?) 

^ And, how U.S. students compare wi .h s 'dents in other counl'.ries in 
how much hc»nework they are required to 
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2. Content — the courses taken by students, or the subject matter 

presented to students, and the substance learned in these courses* 

What the Commission reported concerning content : 

^ That high school students have migrated to the general track from 
vocational and college-prep programs, resulting in a lack of focus in 
their curricula. (Risk , page 18) 

^ That few high school students were taking courses in such subjects 
as advanced math, foreign languages, or geography, even though these 
courses have been offered to them, generally.* ( Risk , page 18) 

^ And that too many of the credits being earned by high school 
students have been in courses in health and physical education, 
out-of-school work experience, remediation, and courses aimed at 
personal service or development, such as training for adulthood and 
marriage.* ( Risk , page 18-19) 

(The Commission addressed the courses taken by high school students 
under two areas — under "Content," concerning the nature of the 
coursework students take or do not take, and under "Standards and 
Expectations," reflecting the rigor of the programs of coursework we 
expect of students. Indicators of the nature of the coursework taken 
by, or presented to, students are discussed here.) 

What the Commission could not report concerning content ; 

® The nature of the courses taken by students in high school.* 

® The content of the courses taken: for example, what students 
study in "General Science" or "World History" in high school, or in 
English in the fifth grade. 

® Whether the set of courses typically taken by students in high 
school has changed over time, beyond shifts among tracks or programs. 

® How the substance of the courses offered to, and taken by, students 
in high school, or the substeuice of subject matter taught in 
elementary school, has changed over time. 

^ Whether the courses taken by students in high school in this country 
differ from the courses taken by high school students in other 
countries. 

^ Whether i and how, the substance of the courses typically offered and 
taken by students in high school, or the substeuice of the subject 
matter taught in elementary school, differs among industrialized 
countries. 



*The Conmission was able to report on students' course-taking patterns 
only because it requested £md supported a one-time survey of the 
transcripts of high school graduates. The information was not then 
readily available among data-reporting activities, and still is not. 
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^ How the content of courses taken by students in high school, or of 
subject matter taught in elementary school, corresponds to what the 
public and its educational leaders feel the schools should teach* 

^ Whether the material in widely-used textbooks is up to date. 



3» Staoidards and Expectations — The requirements and values 
CQimnunicated to students at points in their school careers* 

W hat the Commission reported concerning standards and expectations ; 

^ That, during a period of time when the amount of homework assigned 
to students was declining, grades rose and achievement fell. ( Risk , 
pages 19-20) 

° That students in other countries were required to take three times 
as many advanced mathematics and science courses as students in the 
U.S. ( Risk , page 20) 

^ That states required too few courses in the academic basics of 
mathematics, science and conputer technology, social studies, English, 
and foreign languages. (Risk , page 20) 

° That statewide testing programs aimed almost universally for 
"miiiimal'* competencies, minimums which had become the ""meLximum" 
expectations that were communicated to students. ( Risk , page 20) 

° That many public colleges have chcmged their admission policies to 
accept any high school graduate in their state, serving notice that 
the nature of a student's coursework or his or her grade point average 
in high school make no difference in determining whether he or she 
attends college. (The Commission noted that a trend in this direction 
also had taken place aioong more selective colleges and universities.) 
(Risk , pages 20-21) 

° And that the textbooks and other instructional materials used in 
school demaoid too little of students, having been written down to 
lower reading levels, showing less and less of the influence of 
experienced teachers and scholars, and presenting students with 
academic material that would not be likely to challenge meuiy of them. 
(Risk , page 21) 

What the Commission could not report concerning standards and 
expectations ; 

Whether the courses rec[uired by states for graduation from high 
school have changed over time, and, if so, how. 

Whether local course requirements for high school students differ 
from state requirements, and, if so, what they are and how they 
differ. 

^ Whether local requirements are changing over time, and, if so, how. 
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• The rigor of the school work that is actually required of students 
through assignments, homework, and course tests and quizzes, beyond 
the inferences that can be made from textbooks and achievement tests* 

^ Whether and how the academic demands of school work have changed 
over time, and how the work done by students in this country compares 
with the work required of students in other countries. 

^ Whether, m general, the content of textbooks has become less 
demanding, academically, over time# or whether the books have just 
become easier to read. 

^ Whether state and local testing progreuns are becoming more or less 
demanding, over time. 

^ What standards or expectations grades or teacher testing in subject 
matter communicate to students, and how these teacher standards or 
expectations compare with local or state achievement testing and 
minimal competency testing. 

^ How the standards represented by grades and teacher testing in this 
country compare to grading and subject matter testing of teachers in 
other countries. 

® Whether grading has become "inflated" over time — that is, whether 
teachers are giving higher grades for comparable work, compared with 
grades given in the past. 

® Whether there has been a trena up or iown over time in the use of 
tests or other standards to determine if a student is promoted from 
grade to grade or level to level in school. 

® Whether the school systems of other countries use tests and other 
yardsticks more than we do to determine progress through the school 
system. 

® Whether parents expect more or less from their children in school 
now than they used to expect and communicate. 



4. Teaching — the quality of teaching as a practice and as a 
profession. 

What the Commission reported concerning the quality of teaching . . . 
. . « as a practice : 

® That the aptitude test scores of students going into teaching are 
too low* ( Risk , page 22) 

^ That too little of the teacher preparation program, especially for 
the preparation of elementary school teachers, consists of courses in 
educational methods, and too little in the subjects to be taught. 
(Risk , page 22) 
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^ That teachers are paid too little and have too little responsibility 
for important professional decisions, such as the selection of the 
textbooks they use. ( Risk , pages 22-23) 

^ That there are critical shortages of teachers in certain subjects 
(mathematics, science, and foreign languages) and specialties 
(education of the gifted and talented, of language minority students, 
and of the disadvantaged) . ( Risk , page 23) 

^ And that shortages of teachers in mathematics, science, and English 
are resulting in a large proportion of new teachers hired to teach in 
these areas who are not trained or certified to teach the subjects. 
(Risk , page 23) 

What the Commission could not report concerning the quality of 
teaching . • • 

.. . as a practice ; 

® The capability of our teachers, directly measured, in the 
professional practice of planning and conducting lessons. 

® The pedagogical skill, generally, of our teachers, compared with the 
teachers of the past, or with the classroom skill of teachexs in other 
countries. 

^ The quality of teachers' knowledge of the subject matter they teach, 
measured against notions of what they should know, compared with 
teachers in the past, or compared with teachers in other countries. 

^ The ability of our teachers to teach academic subjects to students 
who vary in background, capability, and interest or motivation; 
whether our teachers are improving in this ability over time; and how 
our teachers compare in this regard with teachers trained in other 
countries . 

^ The relationship between alternative approaches to teacher 
preparation cuid the relative proficiency of teachers in the classroom. 

^ The prevalence or status of different approaches to teacher 
pif^paration that seem to be either successful or unsuccesb'ful. 

® The ability of teachers to handle classroom discipline problems; how 
this ability has been changing over time; and how our teachers compare 
in this regard with teachers in other countries. 

^ The nature and succ<iss of efforts by school systems and states to 
improve the proficiency of teachers through staff development or 
inservice training. 

What the Commission could not report concerning the quality of 
teaching . . • 

o .12 
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• • ' fts a prof assign ; 

^ The range in average teacher salaries among states r school 
districts f and professional specializations. 

^ How the salaries of U.S. teachers compare with those o£ teachers in 
other countries, in terms of relative buying power, and whether the 
buying power of teachers is going up or down in different countries. 

^ Early indicators of the long-range supp.Ty of teachers, such as the 
career intentions of students graduating from high school. 

^ Reasons cited by high school seniors or college students for not 
going into teaching (ie, low salaries, the poor prestige of the 
profession, perceptions of teachers having little professional 
autonomy, poor working conditions, etc.). 

^ The relative importance of various incentives in making teaching 
more attractive to those in the profession, such as higher salaries, 
greater possibility for career growth, more professional autonomy, 
better working conditions and support. 

^ The status of various professional incentives for teachers, both in 
this country over time and in other countries. 

^ Systematic tracking of the demand for, and supply of, teachers 
broken down into various professional specialties, and reported for 
states or regions of the country. 

Trends over time in the standards applied by states to certify new 
teachers and to maintain the certificates of veteran teachers. 

^ The nature of the standards applied by local school districts in 
recruiting and retaining teachers, including the use of different 
approaches for evaluating teacher performance. 

^ How the public perceives the status of teaching as a profession, and 
how this has changed over time. 

^ Perceptions of the status of teaching as a profession in this 
country, compared to its prestige in countries with different 
histories, cultural values, and orgeinizational structures for 
education. 
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5. Leadership and Support — Efforts by the public, policy-makers, and 
parents to support and provide direction to the schools. 

(The Commission on Excellence did not report findings having to do 
with the support and leadership provided to the schools, but the 
CommiBSion did make recommendations in this area, implying the need 
for tho following indicators, which do not seem to be available, now:} 

' Regular reports of the fiscal effort put forth by different 
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countries to support their schools, in terms of school expenditures as 
a proportion of GNP. 

^ Evidence of the range and level of specific efforts to support 
sch ols at the state and local levels, such as levels of expenditure, 
number or proportion of bond issues passed, or changes in state 
appropriations for education. 

^ The range in tax burden for education among states and local school 
districts . 

® The relative cost of meeting different educational demands.* such as 
education of disadvcuitaged, handicapped, or gifted cuid talented 
students . 

® Geographical differences in the costs of education, such as the 
costs among regions of the country or different costs for schooling in 
sparsely-populated areas, cities, and other types of community. 

^ The relationships between cost factors and educational outputs. 

® The involvement of parents and the local community in making 
decisions about school programs, or in contributing to these programs. 

® The nature and level of efforts by parents to monitor and encourage 
the progress of their children in school. 

® The nature and number of cooperative programs beween business and 
the schools. 

® The perceptions of policy-makers and leaders at the national, state, 
and local levels about the quality of the schools and what the 
strengths cuid weaknesses of the schools may be. 

® The nature of school iniprovement efforts being launched at the 
national, state, and local levels, and trends in these efforts over 
time. 



OUTPUTS 

What the Commission reported concerning student achievement ( Risk , 
pages 8-9) : 

® That there had been general patterns of decline in scores on 
standardized achievement tests administered nationwide. 

^ That scores had declined in general on the Scholastic Aptitude 
Tests . 

^ That scores had declined on the College Board achievement tests in 
subjects such as physics and English. 

^ That achievement in science had declined steadily on the National 
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Assessment of Educational Progress from 1969 to 1977. 

^ That there were specific problems of poor (and declining) 
performance in the "higher order" aspects of reading, writing, and 
mathematics on the National Assessment. 

^ That U.S. students fared poorly in international comparison of 
achievement. 

^ That the number and proportion of students scoring 650 or higher on 
the SAT's had dropped. 

^ That most gifted students appear to be achieving at a level below 
their tested aptitude. 

^ That scores on the Graduate Record Examination had declined. 

^ Results of one-time surveys of functional illiteracy which indicated 
that about 23 million adults were functionally illiterate, that about 
13% of school-aged youth were functionally illiterate, and that over 
40% of minority youth were functionally illiterate. 

The Commission also recognized (Risk , page 11) that the average person 
in the U.S. today is provided more schooling (if not better schooling) 
than the average person received a generation ago. This point is 
based on steady increases in the proportions of people completing high 
school, attending college, or completing college over the past 80-100 
years. 

What the Commission could not report concerning student achievement ; 

^ The number or proportion of students nationwide who seem to know 
concepts and principles that would be recognized as basic to the 
academic subjects taught at different levels in school. For example, 
how many students understand and can apply the experimental method in 
science, know the central theme of Moby Dick , or can apply tenets of 
the Bill of Rights to contender ary situations? 

The achievement of today's students in the U.S. compared with 
contemporary students in other countries, as opposed to comparisons 
made 10-15 years ago. 

^ The ability of students in the U.S. to solve complex problems in the 
different academic subjects by finding and interpreting the 
^PP^op^icite information, reasoning analytically, and expressing their 
conclusions effectively. 

^ Trends over time on a comprehensive index of our educational 
productivity that would be more meaningful than SAT scores; for 
example, Wurtz and I (Wurtz and Selden, 1985) have suggested an annual 
* national educational index" based on the product of two numbers: the 
proportion of a common core of academic content objectives that 
students seei.^ to have learned each year, and the percentage of 
students who have complete certain levels of schooling, such as hi ^ 
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school, that year. (Such an index could, and should, also be ad3usted 
for costs and educational effort factors over time and among 
school systems.) 

^ Trends over time in basic measures of educational achievement, such 
as functional illiteracy. 

^ The range in average student achievement among significant political 
unito, such as states or a sample of local school districts. 

^ Trends in achievement over time in states or in local school 
districts facing different challenges and situations. 

(Since the Commission did not consider many individual indicators of 
student participation emd retention in school, and since these are 
areas where existing data are relatively extensive and useful, I will 
not devote space to consider what indicators of participation may not 
be available.) 



Why We Do not Know Some of the Things We Need to Know About Our 
Schools 

Given this prelimin6u:y , cnidely-organized list of some of the 
indicators I believe the Commission could have used, but that I do not 
believe were availcJble to it, let me conclude by considering why these 
indicators may not be available, so we can at least point toward 
developing them in the future. 

First, let me explain the tests that X did and did not apply to 
this list. I have tried to list only indicators that are relatively 
significants that is, they involve variables and emalyses that would 
be particularly useful to educational decision-makers and 
policy-makers. Second, I have relied on my knowledge of the data on 
the educational system that are ana are not available on a regular 
basis. I have not been cible to conduct exhaustive searches to verify 
my understanding in every case. In some cases (not m2uiy, I hope) , I 
may not be informed well enough; in other cases we may agree that 
related data are available, but I will believe that they are not 
fine-grained enough, or regular, enough to be useful. 

INPUTS 

Time . The status of indicators concerning the amount of time 
scheduled and allocated to schooling reveal a problem that will recur 
frequently in this discussion. The length of the school year, the 
length of the school day, and the structure of the school day are set 
by states, more or less officially depending on the state. Local 
school districts and individual schools, public and private, may meet, 
fail to meet, or exceed these standards, so that local practice may 
v£u:y substantially. Only a census of the states and a regular survey 
of local districts would reveal norms and the degree of varieQ^ility 
around the norm for these dimensions, and each brings with it effort 
in terms of data collection and reporting. 

Indicators involving how time is used in school introduce the 
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second problem eLdeitiic to indicators. Even though we know that how 
teachers use scheduled time is important, measuring this variable 
validly and monitoring it over time or among school systems present 
serious technical challenges, and would probably be costly. 
Similarly, measuring and tracking how well students are taught to use 
their own time to enhan^^e learning would be difficult and costly. 

Content . The prcjlem here seems to be the level of detail of the 
information we collect. We are just beginning to monitor the courses 
that are required by states and local school districts for graduation 
from high school. While these efforts have been one-shot projects 
conducted to monitor recent reform initiatives, they could be made 
regular quite easily. The bigger challenge is collecting information 
on what students stud^ below the level of the course: what is 
typically presented to students in each subject at each level of 
school, and how much does this content vary among schools around the 
country? This would require surveys of state and local curriculum 
objectives, content analyses of widely-used textbooks, and surveys of 
teachers to describe what other content and activities they present to 
their students. 

Stcuidards cuid Expectations . Needed here are surveys of state, 
local, and classroom practices in setting, communicating, and 
enforcing academic standards for students. We know how many states 
have minimal competency testing programs, and we recently learned what 
subjects they covar, at what levels they are administered, and how 
thp' are used, in most states. We also know, from occasional surveys, 
how much homework students are assigned. We do not know what 
standards states and local school districts apply to pronote students 
from level to level. We do not know what criteria teachers apply in 
assigning grades cuid how they might be changing over time. We do not 
know how teachers use classroom tests of subject matter. We do not 
know enough about what parents expect of their children in school, or 
how well these expectations are communicated to students. In order to 
obtain this information, we would need new or expanded surveys of 
accreditation bodies, local school administrators, teachers, parents, 
and students. 

Teaching . Indicators of the quality of teaching as a practice 
suffer one, central shortcoming: they are all indirect. We infer the 
competence or professional skill of teachers frc«m aptitude test 
scores, college grades, courses studied, paper-and-pencil qualifying 
examinations, euid conqpliance with certification stwOidards, but not 
from direct measures of the ability of teachers to teach students* To 
measure teaching skill directly, we would re^i cui informed definition 
of the qualities and behaviors that go into good teaching, and then we 
would need to operational ize Uiis definition (or multiple definitions) 
with procedures for observing teachers. With definitions and 
operating procedures (which some states and many local school 
districts are developing in order to evaluate teachers) a national, or 
international, sample of teachers could be observed periodically to 
provide longitudinal euid comparative data on the overall pedagogical 
ability of teachers. With these techniques, we could also refine the 
observational methods to measure special ab.^lities of teachers — the 
ability to teach students with particular characteristics with which 
we are concerned, or to do other, specific aspects of the job of 
teaching. 
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Indicators on teaching as a profession will probably fill in 
quickly. There is a great demand for information on what state and 
local school systems are doing to enhance incentives and working 
conditions for teachers r covering salaries, pay-for-performance 
provisions, career ladders, professional evaluation systems, and steps 
to reduce non-professional duties and stress* The challenges for 
indicators in this area seem to be arriving at standard definitions of 
concepts such as salaries, benefits^ cuid professional incentives, and 
then setting up routine programs and procedures for collecting and 
reporting this information on states cuid local school districts. 

Leadership and Support . The Commission's analysis, and the 
school improvement efforts that came with the reform movement of which 
the Commission was a part, included many recommendations for local 
school leaders and administrators to enhance the involvement and 
support of parents, citizens, businesses and others in the schools. 
To track these efforts and at least their perceived effectiveness, 
surveys would be needed of school administrators, school board 
members, business leaders, and others to determine the extent and 
nature of efforts they have made to develop community support and to 
exeis;ise new levels of leadership for the schools. 



OUTPUTS (Achievement) 

Currently-available achievement indicators are lacking in their 
level of detail. The National Assessment of Educational Progress has 
made a major contribution in providing us with periodic, national data 
at three age levels on student achievement in each of the basic 
academic subjects. What we need beyond this is to break down 
achievement data into greater detail. We need to monitor the specific 
portions of the core, academic subject matter we are concerned with in 
the schools that students know and do not know. NAEP is being 
augmented to assess the status of this kind of specific, 
subject-matter knowledge in the areas of literature and American 
history. Collecting this level of information across subjects on a 
reguleu: basis would permit more effective fine-tuning of educational 
programs at the state, local, and classroom levels, allowing us to 
attend to those parts of the academic program that students do not 
seem to be learning. We cannot do this with the level of detail 
offered by NABP's current design. We also need more detailed sampling 
and reporting, so t.hat outcomes on the Assessment can be reported by 
state and by meaningful types of local school district. Finally, we 
need to collect information on educational programs or efforts that 
are a3sociated with student achievement, in order to begin to 
understand what educational efforts or approaches seem to provide the 
best results; NAEP has begiin to collect this kind of information on 
the schools it samples, and this effort should be refined and 
developed. 

International indicators of achievement suffer mainly from 
infrequency. The lEA studies are on a 10- to 15-year cycle, meaning 
that sometiiiies the most recent available comparative data are very 
old. Five years would seem to be a reasonable time period for this 
cycle, to ensure that comparsions are not erroneously extrapolated 
from situations that no longer exist. Along with a shorter cycle. 
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lEA-type studies should be made more routine. Under the present 
arrangements for conducting the lEA, we cannot count on the studies 
being repeated in the future; each cycle is conducted through private 
organizations in an ad hoc way. The program should be 
institutionalized so that participation in the studies and support for 
them can be relied- upon. 



Summary 

I have attempted to do three things in this paper. The first was 
to establish a structure by which one could talk about educational 
indicators — both those that we already know about and those that we 
may need, but do not have available to us, now. The second was to 
describe some of the indicators that at least one group, the National 
Commission on Excellence in Education, was unable to analyze because 
they did not, and still do not, seem to be available. The third was 
to speculate briefly on what would be involved in developing at least 
some of the import suit indicators that we are missing. 

This paper should really be used a pilot-test for a process that 
should be done more thoroughly and systematically, but it does reveal 
both that there are important kinds of information about education 
that we lack, and that we can identify and address these gaps. 
Venturing forward into these undeveloped areas will take time, effort, 
and money, but few tasks offer as great a potential to give us useful 
tools for improving education. Information is power, and better 
information about education would give us tremendous leverage in 
managing and improving it. 
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